08. Performance Profiler

Step 6. Performance Profiler

It's not enough just to have a new web crawler — you need to understand how its performance stacks up against the legacy code. How else will you be able to prove your implementation is better? Time is money, and cutting costs is a sure way to make a good impression. To do this, you will implement a basic method profiler.

What is a method profiler, you ask? It is a utility that writes down how long method calls took to complete during a particular run of the program. Profilers can get really fancy, but in this case you will be implementing a relatively simple one that writes to a text file like this:

Run at Fri, 18 Sep 2020 02:04:26 GMT
com.udacity.webcrawler.ParallelWebCrawler#crawl took 0m 1s 318ms
com.udacity.webcrawler.parser.PageParserImpl#parse took 0m 2s 18ms

Implementing the Profiler

Everything you need is in the src/main/java/com/udacity/webcrawler/profiler/ folder.

The profiler will record the running times of different method invocations. Now that the crawler runs in parallel, this could happen concurrently from multiple different threads. The ProfilingState class has already been implemented for you to be thread-safe.

You will be writing a method interceptor that intercepts method calls annotated with the @Profiled annotation. To create classes whose methods are intercepted, you will implement the Profiler utility class. This class will "wrap" the to-be-profiled objects in a dynamic proxy instance.

Implementation will include the following steps:

  • Fill in ProfilerImpl.java. Reading Java's Proxy documentation will be very helpful.

  • Fill in ProfilingMethodInterceptor.java. The invoke method should check whether the passed Method is annotated with @Profiled. If the method has the annotation, the interceptor should use the injected java.time.Clock to measure the duration of the method invocation, and then record it to the ProfilingState.

Your interceptor should always return the same value (or throw the same Throwable) as the method that was intercepted. When implementing this, there are some common "gotchas" to look out for:

  • Think carefully about how the proxy should behave for the java.lang.Object#equals(Object) method. Reading the InvocationHandler documentation will be very helpful.

  • Special care should be taken to correctly handle exceptions thrown by the intercepted method. If the intercepted method throws something, the proxy should always throw the exact same thing. Make sure your proxy doesn't accidentally throw a UndeclaredThrowableException instead. Also remember that, even if it throws an exception, any @Profiled method should still have its running time recorded!

(Note: Due to limitations of the Java language, objects must be manually "wrapped" (using Profiler.java) to be profiled. The starter code already does this for you! Thanks, dependency injection! More sophisticated AOP frameworks sometimes use bytecode generation to avoid this kind of boilerplate.)

Once you are done, run the unit tests to check your work:

mvn test -Dtest=ProfilerImplTest

If you have been following the instructions in order and have already implemented the web crawler, you should now be able to run all the tests at once to make sure they pass:

mvn test

Output

Finally, the profile data should be written to a file defined by the profileOutputPath in the crawl configuration. Locate the final TODO in WebCrawlerMain.java and take care of it. The code for this should be very similar to the code you wrote for the crawl result file, but this time use the profileOutputPath for the file name.

(Optional) Profiler Enhancements

If you want to make your profiler stand out (and also be significantly more useful), try enhancing it in the following ways:

  • In addition to the basic information it already records, have the compiler also record the thread ID of the thread in which the method invocation happened.

  • Make it remember and record the number of times a particular method was called.

Next Concept